(Re)Design Considerations for Scalable Large-File Content Distribution
نویسندگان
چکیده
The CoBlitz system was designed to provide efficient large file transfer in a managed infrastructure environment. It uses a content distribution network (CDN) coupled with a swarm-style chunk distribution system to reduce the bandwidth required at origin servers. With 6 months of operation, we have been able to observe its behavior in typical usage, and glean information on how it could be redesigned to better suit its target audience. At its heart, this paper describes what happens when a plausible conceptual design meets the harsh realities of life on the Internet. We describe our experiences improving CoBlitz’s performance via a range of techniques, including measurement-based feedback, heuristic changes, and new algorithms. In the process, we triple CoBlitz’s performance, and we reduce the load it places on origin servers by a factor of five. In addition to improving performance for CoBlitz’s users, we believe that our experiences will also be beneficial to other researchers working on large-file transfer and content distribution networks.
منابع مشابه
Deploying Large File Transfer on an HTTP Content Distribution Network
While HTTP-based Content Distribution Networks have been successfully used to serve Web users, several design and operational issues have prevented them from being used for the scalable and efficient transfer of large files. We show that with a small number of changes, supporting large file transfer can be efficiently handled on existing HTTP CDNs, without modifying client or server software. T...
متن کاملFastReplica: Efficient Large File Distribution Within Content Delivery Networks
In this work, we consider a large-scale distributed network of servers and a problem of content distribution across it. We propose a novel algorithm, called FastReplica, for an efficient and reliable replication of large files in the Internet environment. There are a few basic ideas exploited in FastReplica. In order to replicate a large file among n nodes (n is in the range of 10-30 nodes), th...
متن کاملScalable and efficient file sharing in information-centric networking
Internet usage has been increasingly shifting toward massive content retrieval and distribution. To satisfy the demand, information-centric networking (ICN) approaches have emerged. These approaches have common features such as in-network caching and the decoupling of content and location. Such a content-centric communication model enables efficient content delivery. In this article, we investi...
متن کاملA Case for P2P Delivery of Paid Content
P2P file sharing provides a powerful content distribution model by leveraging users’ computing and bandwidth resources. However, companies have been reluctant to rely on P2P systems for paid content distribution due to their inability to limit the exploitation of these systems for free file sharing. We present TP2, a system that combines the more cost-effective and scalable distribution capabil...
متن کاملStructured P2P technologies for distributed command and control
The utility of Peer-to-Peer (P2P) systems extends far beyond traditional file sharing. This paper provides an overview of how P2P systems are capable of providing robust command and control for Distributed Multi-Agent Systems (DMASs). Specifically, this article presents the evolution of P2P architectures to date by discussing supporting technologies and applicability of each generation of P2P s...
متن کامل